Anycast is a network addressing and routing methodology for routing packets from a sender to the closest of several potential recipients that are each identified with the same address. Anycast is built upon the Border Gateway Protocol (BGP). BGP advertisements announce that the same address is available at different topological locations throughout a network (i.e., Internet). Routers receiving the advertisements continually cull their routing tables to identify the shortest path to one such Anycast destination.
Anycast lends itself for use with a distributed platform. A distributed platform operates a plurality of geographically distributed points-of-presence (PoPs) for the purpose of serving content or providing services from each of those PoPs to different sets of users in a distributed manner. Each PoP typically includes at least two collocated servers. The multiple servers of a given PoP host the same or different content and/or perform the same or different services. This ensures that the PoP has adequate resources for serving the content or for providing the various services while also ensuring redundancy and balanced load distribution across the PoP.
By using Anycast in combination with such a distributed platform, user requests for content or services can be routed to the distributed platform PoP that is closest to the requesting user. In so doing, the Anycast identified PoP can deliver the content or services in an optimal manner, in part, because of the proximity to the user. The proximity to the user reduces the number of hops or network nodes over which the content or service travels in order to reach the user. Each additional hop or node increases the potential for added latency, buffering, and packet loss. A content delivery network (CDN) is an example of a distributed platform that benefits from the routing efficiencies of Anycast.
However, some legacy routing and control mechanisms of underlying networking protocols were never designed to work with such Anycast reliant distributed platforms. As a result, aberrant behavior may occur in these Anycast reliant distributed platforms. FIG. 1 provides an exemplary case of legacy control messaging producing aberrant behavior in an Anycast reliant distributed platform.
FIG. 1 presents a message exchange between a client 110 and an Anycast identified PoP 120 of a distributed platform, wherein the aberrant behavior results from a control message that is issued by an intermediary node 115 in the path connecting the client 110 to the PoP 120. In this figure, the PoP 120 includes a core router 125, two directors 130 and 135, and two servers 140 and 145, wherein the directors 130 and 135 are load balancing devices distributing loads across the servers 140 and 145. It should be noted that the same aberrant behavior illustrated by FIG. 1 may occur if the PoP 120 was horizontally or vertically scaled to include more core routers, directors, or servers.
The message exchange of FIG. 1 commences with the client 110 submitting (at 150) a request for content to an Anycast address of the distributed platform. Anycast routing delivers the request to the PoP 120 and more specifically, to the core router 125. The core router 125 performs a simple hash of the source and destination addresses specified in the request header to select director 130 to process this request and any subsequent messages identifying the same source and destination addresses. The core router 125 then forwards (at 155) the request to the selected director 130.
The director 130 is tasked with selecting (at 160) a server from the set of servers 140 and 145 to respond to the user request. The director 130 first performs a session table lookup to determine if a prior session is established for the request and if a server has already been selected to handle the session. In this case, no prior session exists. The director 130 selects server 140 based on a hash of the name and/or path of the content specified in the request and/or addressing information from the request header. The director 130 creates a session table entry such that future inbound packets from the client 110 that are associated with the session are properly routed to the correct server 140. The session can be identified by any of the requested content name, requested content path, or a source and destination address combination, wherein the term address refers to any one or more of a Media Access Control (MAC) address, Internet Protocol (IP) address, and application or transport layer protocol port number. In this example, the session table entry identifies the current session with a source IP address, a source port, a destination IP address, and a destination port. After creating the session, the director forwards (at 165) the request to the selected server 140.
The server 140 processes the request and begins serving (at 170) the requested content to the client 110 in response. However, the intermediary node 115 in the network path between the client 110 and the server 140 submits (at 175) a control message back to server 140 or the Anycast address specified by the original request. Control messages can be used for a variety of reasons. For the current example, the control message indicates that the intermediary node 115 cannot support the manner in which the server 140 sends the content. One such control message indicates that the intermediary node 115 does not support packets of the size submitted by the server 140 and that the server 140 should retransmit the packets with a smaller size. Such a control message can be an Internet Control Message Protocol (ICMP) message indicating that the packet(s) exceeds the maximum transmit unit (MTU) supported by the intermediary node 115. It is important to note that the control message source address is the intermediary node's 115 address and not the address of the client 110. Also, in some instances, the control message destination address may identify the Anycast address and not the unique IP address for the actual server 140 within the PoP 120 that transmits the offending packets that cause the intermediary node 115 to issue the control messages.
Anycast routing delivers the intermediary node 115 control message back to the core router 125 (provided that Anycast routing continues to identify the PoP 120 of the core router 125 as the closest distributed platform PoP reachable using the Anycast address). The core router 125 may select (at 180) an incorrect director that has no knowledge of the client's 110 active session (i.e., director 135) as the selection is based on a hash of the intermediary node 115 address that is provided in the control message header rather than the client 110 address. If the control message is passed from the core router 125 to director 135 (i.e., the incorrect director), the director 135 is unable to identify the server 140 or the session to which the control message relates. The session table maintained by director 135 will not have an entry for the address combination in the control message header or addressing from an offending packet header which may be included as part of the control message. In this case, director 135 will ignore the control message such that the control message does not arrive at the appropriate server 140 sending the offending packets. Until the server 140 receives the control message, the server 140 will continue to send the offending packets and the intermediary node 115 will continue to block or drop those packets, thus preventing the client 110 from receiving the requested content.
The above depicts one scenario where aberrant behavior results from legacy control mechanisms in an Anycast reliant distributed platform. In summary, the aberrant behavior results when the session implicated by a control message cannot be identified from the control message.
In most cases, the operational behavior of the core router 125 leading to the improper routing of the control messages amongst a set of directors cannot be modified. This can be due to a variety of reasons including the core router's use of proprietary algorithms to perform the message distribution across the directors. In addition to being proprietary and unchangeable by any one other than the core router manufacturer, the algorithms may be directly incorporated into inaccessible or non-configurable application specific integrated circuits (ASICs) or other integrated circuits of the device that cannot be modified by the distributed platform.
Similar aberrant behavior can be observed in different distributed platform architectures that rely on Anycast routing. The aberrant behavior can also be observed when different control messages, error messages, or status messages besides the exemplary case described above are issued from intermediary network nodes to a distributed platform that uses Anycast routing. Accordingly, there is a need to resolve the aberrant behavior that occurs from the inability to directly identify from a control, error, or status message, the connection or session to which the message relates. Stated inversely, there is a need to adapt legacy control mechanisms and various control, error, or status messages, such as ICMP and MTU too large messages, for Anycast reliant distributed platforms.